Repository navigation
Move ARM64 Helix container hosts to Azure Linux 4 - #136387
Merged
steveisok merged 2 commits intoOct 24, 2026
Merged
Conversation
Update only container-host queues while preserving payload images, aliases, routing, ARM32 hosts, and standalone ARM64 VM coverage. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 96a5c84b-34cb-4531-b8cf-e6502299ec1d
|
Azure Pipelines: Successfully started running 5 pipeline(s). 11 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
Contributor
|
Tagging subscribers to this area: @dotnet/runtime-infrastructure |
jkotas
approved these changes
Oct 24, 2026
Contributor
There was a problem hiding this comment.
🔵 Needs a closer look
Human sign-off is needed on native ARM64 execution evidence, including mappings outside standard CI coverage, because static checks do not establish host/container compatibility.
No open issues 🎉
What changed in this PR
Moves native ARM64 Helix container hosts from Azure Linux 3 to Azure Linux 4, extending the earlier x64 migration without changing the distributions tested inside containers.
Changes:
- Updates 11 host mappings: eight public and three internal.
- Preserves payload images, digests, aliases, and lane conditions.
- Leaves ARM32 container mappings, standalone ARM64 VMs, and build agents unchanged.
| File | Description |
|---|---|
eng/pipelines/libraries/helix-queues-setup.yml |
Moves Ubuntu, Azure Linux 3, and Alpine ARM64 container hosts to AL4. |
eng/pipelines/installer/helix-queues-setup.yml |
Moves Ubuntu and Alpine ARM64 container hosts to AL4. |
eng/pipelines/helix-platforms.yml |
Updates public/internal hosts for Alpine ARM64 latest/oldest aliases. |
eng/pipelines/coreclr/templates/helix-queues-setup.yml |
Updates public/internal hosts for Alpine ARM64 tests. |
Review effort: Balanced
3 tasks
steveisok
added a commit
that referenced
this pull request
Nov 5, 2026
<!-- --> ## Summary Prepare the **H2 ARM32 host-only candidate** from [#136313](#136313), following the native ARM64 container-host changes merged in [#136387](#136387). Change only the host component `AzureLinux.3.Arm64[.Open]` to `AzureLinux.4.Arm64[.Open]` for existing Debian 13 and Alpine 3.24 ARM32 containers: | File | Host mappings | |---|---:| | `eng/pipelines/coreclr/templates/helix-queues-setup.yml` | 4 | | `eng/pipelines/libraries/helix-queues-setup.yml` | 1 | | `eng/pipelines/installer/helix-queues-setup.yml` | 1 | | `eng/pipelines/helix-platforms.yml` | 8 | The 14 replacements comprise eight Debian and six Alpine mappings, with eight public and six internal queue strings. Parenthesized aliases, public/internal routing, image tags, all six digest pins, existing tag-only aliases, payload architectures, ABI/sysroots, cross-build inputs, conditions, queue groups and lane counts are unchanged. All 11 native ARM64 container-host mappings from #136387, both bare AL3 ARM64 VM entries, other platforms, build agents and SuperPMI are preserved. This is not an ARM32 image upgrade, distro-coverage change, test conversion to ARM64, or host-tooling/provisioning change. It does not close #136313 or declare H2, S1 or S2 complete. ## Compatibility status and merge requirement **Actual ARM32 execution on the deployed Azure Linux 4 ARM64 Helix hosts is unverified. This is a candidate, not a proven-compatible deployment. Do not merge based on static checks or ARM64 queue availability alone.** Read-only [Helix queue metadata](https://helix.dot.net/api/2019-06-17/info/queues) advertises gallery image `MicrosoftAzureLinux/azurelinux-4/4-arm64/4.0.2026091402` and an ARM64 Alpine smoke image. The exposed artifact metadata does not establish ARM32 execution or installed QEMU/binfmt support. The [upstream AL4 kernel configuration](https://github.com/microsoft/azurelinux/blob/1035368c970cbed17ab06a8006b6c57b25fdc856/specs/k/kernel/6.18-aarch64-azl.config#L527) enables compat ELF support, but that is not proof of the deployed kernel, CPU AArch32 EL0 capability, or emulation setup. Native execution requires suitable CPU/kernel support; otherwise transparent execution requires an already-provisioned emulator and appropriate binfmt registration/interpreter access. No such configuration is changed here. A bounded sample of 100 recent public ARM32 Helix jobs contained no AL4 ARM64 execution. Successful existing [Debian ARM32](https://helix.dot.net/api/2019-06-17/jobs/b66ca86e-904b-4af4-917c-93f808a1ab67/details) and [Alpine ARM32](https://helix.dot.net/api/2019-06-17/jobs/7f9bae7f-55f8-42d4-9e0a-804d67b96323/details) jobs ran on AL3, not AL4. Neither those passing jobs, kernel `CONFIG_COMPAT`, native ARM64 success, nor `Architecture=ARM64`/`IsAvailable=true` establishes ARM32-on-AL4 compatibility. **No concrete incompatibility has been established either.** Actual Debian and Alpine ARM32 execution on AL4 hosts is required before merge. Confirm the resolved host queue, unchanged ARM32 container, and successfully executed ARM32 workloads, rather than submission-only or ARM64 results: - [ ] `runtime`: `build_linux_arm_checked_CoreCLR_ReleaseLibraries` — Debian ARM32 runtime innerloop, via the CoreCLR queue template (`eng/pipelines/runtime.yml:663-703`). - [ ] `runtime`: `build_linux_musl_arm_checked_CoreCLR_ReleaseLibraries` — Alpine ARM32 libraries on Checked CoreCLR, via the CoreCLR queue template (`eng/pipelines/runtime.yml:704-743`). - [ ] `runtime-extra-platforms`: `linux_arm` Release `Libraries_Release_CoreCLR` — the separately gated Debian ARM32 libraries mapping (`eng/pipelines/extra-platforms/runtime-extra-platforms-other.yml:15-44`). Ordinary `runtime` PR validation does not exercise this libraries-template entry. These are existing routes, not new lanes. No manual CI trigger/retry or Helix submission has been performed or is requested by this change; any manual execution requires separate authorization. Additional existing coverage includes `runtime-coreclr outerloop` Checked JIT runs with `platformGroup: all` for both ARM32 platforms, and scheduled libraries outerloop Debian ARM32 runs with `includeAllPlatforms: true`. ## Coverage gaps - **Internal queues:** public CI does not validate the internal hosts. CoreCLR explicitly selects non-`.Open` Debian/Alpine entries; libraries and installer senders strip `.Open` for internal projects. Those routes are preserved, but no internal execution evidence was obtained. `runtime-official.yml` is publication-only, not an ARM32 test route. - **Installer:** the ARM32 mapping has no current active caller. The three installer queue-template invocations in `runtime.yml` select other platforms. Ordinary `runtime` or extra-platforms success cannot validate ARM32 installer execution. - **Central aliases:** only `helix_linux_arm32_oldest` has in-repository consumers: cDAC dump/stress/xplat routes in `runtime-diagnostics`, plus optional SOS ARM32 selection (excluded from default `sosPlatforms`). These are not covered by ordinary `runtime`/extra-platforms. The remaining seven aliases, including all internal and all Alpine ARM32 aliases, have no in-repository consumers. No coverage expansion or host support installation is included to hide these gaps. Closing them requires separately approved work; public CI success must not be described as exercising every changed mapping. ## Validation - **Passed:** `python3 "$SESSION_FILES/verify-arm32-hosts.py"` using existing PyYAML 6.0.3. The session-local assertion script checks duplicate-key-rejecting YAML parsing, parsed structure equality modulo the permitted host transformation, exact full-file byte equality against that transformation, all 14 replacements, unchanged payload alias/image/digest bytes, preservation of all 11 landed native ARM64 mappings and both bare AL3 ARM64 entries, and the absence of unrelated tracked changes. - **Passed:** `git diff --check`. - **Passed:** read-only MCR tag/manifest/config queries confirmed both existing pinned payloads are `linux/arm/v7`; no Docker commands or image operations were performed. Existing unpinned aliases remain floating tags. - **Not run:** product builds/tests (queue-metadata-only change), Azure Pipelines template expansion, or actual ARM32 execution on AL4 Helix hosts. No manual CI/Helix jobs, Docker operations, provisioning, kernel/binfmt changes, or emulation installation were performed. See the [OS onboarding guide](https://github.com/dotnet/runtime/blob/main/docs/project/os-onboarding.md) for the container-host testing model. > [!NOTE] > This pull request description and the code changes were prepared with GitHub Copilot. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: b085338b-95ae-40c2-a1e0-d77e477f5cfc
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Native ARM64 container-host equivalent of #134819: move the physical Helix host queues from Azure Linux 3 to Azure Linux 4 without moving standalone test VMs or changing payload distro images.
AzureLinux.3.Arm64.OpenAzureLinux.4.Arm64.OpenAzureLinux.3.Arm64AzureLinux.4.Arm64The 11 host-only replacements are limited to four YAML files:
eng/pipelines/coreclr/templates/helix-queues-setup.ymllibraries/helix-queues-setup.ymlinstaller/helix-queues-setup.ymlhelix-platforms.ymlEvery descriptive alias, registry/image tag/digest, distro/version/architecture identity, lane condition, queue variable/group and lane count remains unchanged. In particular,
(AzureLinux.3.0.ArmArch.Open)and itsazurelinux-3.0-helix-arm64v8image remain Azure Linux 3 coverage; only their container host moves.All 14 ARM32-on-ARM64 container mappings and both standalone CoreCLR ARM64 VM mappings retain their existing hosts. Build agents, cross-build images, SuperPMI, GitHub Actions and other platforms are unchanged. The standalone AL4 VM
libmsquicprerequisite is not bypassed by this change.Related to #136313, H1. This PR does not close the tracking issue or declare H1, S1 or S2 complete.
Validation
Static checks on commit
8869ad3d623ff1910b927f74e46adf995debac4b:git diff HEAD^ HEAD --check- passed.ruby -ryaml -e 'ARGV.each { |path| YAML.load_file(path); puts "PASS #{path}" }' eng/pipelines/coreclr/templates/helix-queues-setup.yml eng/pipelines/helix-platforms.yml eng/pipelines/installer/helix-queues-setup.yml eng/pipelines/libraries/helix-queues-setup.yml- passed; all four files parsed.mainreports one commit, four changed files and 11 additions/11 deletions.Refreshed the live Helix queue catalog on 2026-10-23:
azurelinux.4.arm64azurelinux.4.arm64.openQueue availability does not establish capacity or native workload/container/kernel compatibility. No product build, native Helix execution, Docker activity, manual CI trigger/retry or Helix submission was performed for this metadata-only change.
Required native CI coverage and existing gaps
Standard CI selects only part of the changed mappings; it must not be treated as complete coverage of this PR:
runtimeincludes CoreCLR-selected Alpine ARM64 coverage and libraries Azure Linux 3 ARM64 coverage. Confirm actual execution on the new hosts, including the applicable public/internal routes.runtime-extra-platformsselects Ubuntu ARM64 libraries coverage, but its current platform matrix does not listlinux_musl_arm64.includeAllPlatforms=truecoverage, such as the scheduled rolling libraries outerloop path. A normal extra-platforms run alone does not exercise it.isExtraPlatformsBuild=trueorincludeAllPlatforms=true.No coverage wiring is expanded in this PR. Native execution evidence and merge remain pending; static checks and queue availability do not complete the host migration.
Reference: OS onboarding guide.
Note
This PR description was generated with GitHub Copilot.